ELECTRONIC EQUIPMENT AND POINTER DISPLAY METHOD 

BACKGROUND OF THE INVENTION 



The present invention relates to a man-machine interface 
technique, and more particularly to a graphical user interface which 
makes use of a plurality of pointing 



In a man-machine interface of a computer system, the GUI 
(Graphical User Interface) which makes use of a pointing device has been 
widely used. In the general GUI, windows, various kinds of objects and 
a pointer which corresponds to the pointing device and the like are 
displayed on a screen and inputs are received from the pointing device 

manipulated by a user. 

Recently, along with the progress of the improvement of the user 
interface in the computer system, various ideas have been proposed to 
enable a manipulation with a feeling close to the real world even in the 
GUI. For example, at the time of deleting unnecessary files, drugging 
and dropping of these files into an icon indicating a litter bin is performed 
as if litters are actually thrown into a litter bin in a real space. 

* * 
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However, in spite of the fact that an environment very close to the 
real space is realized in the virtual space on the computer in the 
above-mentioned manner, the number of pointing device that the user 
uses is usually only one. Accordingly, the feeling of the user, even with 



respect to a work which the user uses both hands to handle in tine real 
space, becomes the feeling of performing the work with a single hand 
when the pointing device is used on the GUI and hence, the user receives 

the cumbersome feeling. 

Accordingly, it is an object of the present invention to provide a 
technique on GUI which ensures a manipulation with a feeling close to a 
feeling at the time of performing the manipulation in a real space. 

To achieve the above-mentioned object following processing is 
performed according to the present invention. That is, a first input 
signal is received from a first pointing device and a display position of a 
first pointer which is displayed on a display device is determined based 
on the first input signal. A second input signal is received from a second 
pointing device and a display position of a second pointer which is 
displayed on the display device is determined based on the second input 
signal. 



BRIEF DESCRIPTION OF THE DRAWINGS 
Fig. 1 is an explanatory view showing the whole constitution of a 
computer system to which a mode for carrying out the present invention 

is a^ _ 

Fig. 2 i9 an explanatory view showing an example of a first 
manipulation of the present i 

Fig. 3 is an explanatory view showing an example of a second 

manipulation of the present i 

Fig. 4 is an explanatory view showing an example of a third 

manipulation of the present invention. 



Figs. 5A to 5B are explanatory views showing an example of a 
fourth manipulation of the present invention. 

i K s. 6A to 6C are explanatory views showing an example of a fifth 



manipulation of the present invention. 

ig. 7 is an explanatory view showing an example of a sixth 



manipulation of the present invention. 

Fig. 8 is an explanatory view showing an example of a seventh 

manipulation of the present invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
Embodiments to which the present invention is applied are 
explained in conjunction with attached drawings hereinafter. 

Fig. 1 shows a computer system according to this embodiment. 
This computer system includes pointing devices 1 (la, lb), an electronic 
equipment 2 and a display device 3. The electronic equipment 2 includes 
a connecting portion which connects the electronic equipment 2 with the 
pointing device la, a connecting portion which connects the electronic 
equipment 2 with the pointing device lb, and a connecting portion which 
connects the electronic equipment 2 with the display device 3 respectively. 
These connecting portions are not shown in the drawing. The pointing 
devices la, lb and the display device 3 are connected to corresponding 



connecting portions, respectively. 

In this computer system, it becomes possible to provide a plurality 
of pointing devices 1. In this embodiment two pointing devices (la, lb) 
are provided. A user of this system is capable of simultaneously 
manipulating respective pointing devices 1 using both hands. Each 
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pointing device la, lb includes a mechanism for receiving an input which 
is provided for displacing a pointer displayed on a display screen 31 (see 
Fig. 2) of the display device 3 and a button, although they are not shown 
in the drawing. The pointing device 1 may be, for example, a mouse or a 

5 track ball or the like. 

The electronic equipment 2 includes a CPU (Central Processing 

Unit), a memory and the like not shown in the drawing, and 
following functions by executing a computer program. That is, the 
electronic equipment 2 includes input receiving parts 21 (21a, 21b), a 
10 pointer control part 22, an application 23 and a display control part 24 as 

inner functions thereof. 

The input receiving parts 21 receive inputs from the pointing 
devices 1. In this embodiment the input receiving parts 21 are 
comprised of two parts (21a, 21b). Respective input receiving parts (21a, 
15 21b) are made to correspond to the pointing devices la, lb. Respective 
input receiving parts 21a, 21b respectively receive inputs from respective 
pointing devices la, lb. To be more specific, the input receiving parts 21 
the displacement information of pointers and the button 
on information from the pointing devices 1 at a fixed period 
(for example, 1/60 second). The displacement information is, for 
example, comprised of information which indicates a relative 
displacement quantity on the display screen 31 (see Fig. 2 and Fig. 3) 
which constitutes a two-dimensional plane, while the button 
manipulation information is, for example, comprised of information 
which indicates whether the buttons of the pointing devices 1 are in the 
pushed state (ON) or in the non-pushed state (OFF). 
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Here, the number of the input receiving parts 21 is not always as 
same as the number of the pointing devices 1. For example, one input 
receiving part 21 may receive inputs from a plurality of pointing devices 
1. 

The pointer control part 22 obtains the displacement information 
and the button manipulation information which the input receiving parts 
21 receive. The pointer control part 22 holds the positional information 
of pointers 41, 42 (see Fig. 2 and Fig. 3) which are displayed on the 
display screen 31. These positional information are, for example, 
respectively held in the form of XY coordinates on this plane respectively 
using the display screen 31 as a two dimensional X-Y plane. The pointer 
control part 22 calculates the XY coordinates of respective pointers after 
displacement based on the XY coordinates of the pointers 41, 42 and the 
displacement information. The calculated coordinates values after 
displacement are outputted to the display control part 24 as the positional 
information of the pointers 41, 42. Further, this positional information 
and the button manipulation information received from the input 
receiving parts 21 are outputted to the application 23. 

The display control part 24 controls the display device 3 and 
displays an object or the like on the display screen 31. To be more 
specific, the display control part 24 receives the positional information of 
the pointers from the pointer control part 22 and displays the pointers 41, 
42 at the designated positions. Further, upon receiving the 
instruction from the application 23, the display control part 24 displays an 
instructed object or the like on the display screen 31. 

The application 23 performs various processing. Considered here 




is the application 23 which executes a given processing based on the 
instructions from the pointing devices 1 and displays various graphics, 
texts and the like on the display screen 31 is considered. Hereinafter, a 
virtual object or the like which the application 23 displays on the display 

5 screen 31 is called "an object". 

For example, a case in which the application 23 which manipulates 
the object on the display screen 31 as shown in Fig. 2 and Fig. 3 is 
considered. In Fig. 2, a virtual object 32, a color pallet 33 for selecting 
colors and the pointers 41, 42 are displayed on the display screen 31. In 

LO Fig 3, a virtual object 34 and the pointers 41, 42 are. displayed on the 
display screen 31. Here, the pointer 41 and the pointer 42 are 
respectively made to correspond to the manipulation of the pointing 
device la and the manipulation of the pointing device lb. 

The application 23, with respect to the objects such as the virtual 

15 objects 32, 34 and the color pallet 33 and the like on the display screen 31, 
holds information indicative of the kinds of objects, information 
indicative of the shapes of objects and the like and information indicative 
of the positions of the objects and the like respectively. Based on these 
information, the application 23 performs the drawing instruction of the 

20 objects to the display control part 24. 

Further, the application 23 receives the positional information of 
the pointers 41, 42 and the button manipulation information from the 
pointer control part 22. Then, based on the positional information of the 
pointers 41, 42, the application 23 grasps the positional relationship with 

25 the objects which have been already displayed. That is, the application 
23 judges whether the pointers 41, 42 are overlapped to the objects which 
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ho ve been ..ready delayed or net and determines the position where the 
pointers 41, 42 are overlapped to the objects when the pointers 41, 42 are 

overlapped to the objects. Further, when the pointer. 41, 42 are 

overlapped to the existing objects, the application 23 judges the presence 
or the absence of the instruction from the pointing devices 1 based on the 
outton manipulation miction. When the application 23 receives the 
instruction of the pointing device. 1, the application 23 periods a given 

processing. 

For example, in the first manipulation example .hown in Fig. 2, the 
virtual object 32 and the color pallet 33 are displayed on the display 
.creen 31. The user move, the pointer, 41, 42 to respective positions 
shown in the drawing by manipulating the pointing device, la, lb. That 
is the pointer 41 indicates a right upper portion 32a of the virtual object 
32 and the pointer 42 indicate, one color region 33a out of the color pallet 

Here, when the user pushes the button of the pointing device la, 

the application 23 is informed of the fact that the button of the pointing 

device la is pushed as the button manipulation information. Based on 
tni. induction, the application 23 perform, a given processing for 
.electing the virtual object 32 as a subject object 

Then, when the user pushes the button of the pointing device lb. 
the button manipulation information, which implies the button of the 
pointing device lb .. P u.hed, is communicated to the application 2* 
Accordingly, the color of a region 33. which is indicated by the potnter 42 
is .pecified out of the color paliet 33. As a result the appl.cat.on 23 
nonets the display centre, part 24 such that the color of the virtual 
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object 32 which is the seized object is changed to the specified color 
(color which the region 33a indicates). 

Further, in the example shown in Fig. 3, the virtual object 34 » 

displ ayed on the display screen 31. In response to the manipulation of 
the user, the application 23 performs the processing to rotate the virtual 
object 34 about a designated point which works as the center. 
Hereinafter, a manipulation example to perform this rotattng 

manipulation is explained. 

The user manipulate, the pointing device la such that the pointer 

_ ~. m t-> ^ ^j-l^ near 
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41 indicates a point 35 inside the virtual object 34. Further, the user 
manipulates the pointing device lb such that the pointer 42 indicates a 
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point outside the virtual object 34. 

Here, when the user pushes the button of the pointing device la, 
the button manipulation information, which implies the burton of the 
pointing device 1. is pushed, is communicated to the application 23. 
Accordingly, the point 35 inside the virtual object 34 is determined as the 

«™m«mt and the application 23 perforins a 
central point of the rotary movement ana ine »yy 

*. a ~n».t« ine tKe central point of the rotation, 
given processing to determine tne centra* y 

Then, the user moves the pointing device lb such that the pointing 
device lb draws an arc while pushing the button of the pointing device lb. 

.. i» -.eoives the button manipulation 

Accordingly, the application 23 receives tn 

information on the pointing device lb and the positional information of 
*. pointer. Here, on the display screen 31, the pointer 42 move, along 
an .reshaped arrow A. Accordingly, the application 23 instructs the 
dis p.ay control part 24 such that the virtual object 34 i. subjected to the 
ro t.ry movement about the point 35 in response to the movement of the 
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P ° imer L the provision of two pointing devices, the user can perform 
th e m.nipuiation of the ob,ect on - dispLy screen 3. both hands 

as described previous*. manlpul auon ex.mp.es. 

In the above-mentioned first and secon r 

,• «„. 23 does not differentiate the pointer 41 and the pointer 42 
the application 23 aoes not 

k ther That is, the role of the pointer 41 and the role of the 

' According ly, the function which is 

pointer 42 can be exchanged. Accordingly, 

performed by using the pointer 41 can be performed by the ^ 
while the function which is performed by using the pointer 42 can 
performed by using the pointer 41. 

F n-\ m av differentiate the 

On the other hand, the application 23 may d 

pointer. 41. 42 from each other, in rhis case, functions which respec.ve 
pointers 41. 42 can perform are preUminarily allocated to «J - 
pointers 41. 42. Por exampie. one pointer functions as a 
pointer 41 in the firs, and the second m.nipuiation example) - 
Lecfs an o^ec, which becomes a sub.ec. or speciHes a P"*^** 
cornea the reference, whiie the other pointer functions as a pointer ^ 
poster 42 in ft. first and the second manipulation example .) wKkH 
, performs the instmCon of the m.nipuiation using the se.ected o b) ec, or 

•ti-A position or the like as the reference, 
the specified position or . Hf| foT 

. , 95 cutouts an ldenoner ror 

Here, the pointer control part 22 outpux 

i' h'oti m alone with the positional 
id enufying the pointer to the apphcation 23 along 

information and the button manipulation information. «he app 
miormaw receives the instruction is 

* 23 the pointer from which the application 23 receives 
■ 5 23/ ^ P . „ The application 23 performs the 

preliminarily set for each processing. The apphca 
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determination using the identifier of the pointer along with the positional 
information and the button manipulation information. 

Further, .he present invention is applicable to various 
manipulation example beside, the above-mentioned m.nipu,ation 
examples. For example, such other manipulation examples are shown in 
Fig. 4 to Fig. 8. 

The third manipulation example shown in Fig. 4 is an example 
which changes the size of an air brush in a painting tool. That is, the 
pointer 41 selects a location to which the air brush is sprayed. Here, a 
slide bar 51 is manipulated by using the pointer 42 so as to enlarge or 
shrink a virtual circle 52 which indicates the range in which the air brush 
is sprayed thus adjusting the magnitude of the air brush. 

The fourth manipulation example shown in Figs. 5A and 5B is an 
example which performs the deformation processing of the object That 
is two points inside a circular object 53 indicated by Fig. 5A are 
designated by the pointer 41 and the pointer 42 and they are pulled in the 
left and the right directions so as to deform the object 53 into a shape 
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indicated by Fig. 5B. 

The fifth manipulation example shown in Figs. 6A to 6C is an 
example which designates a start point and an end point. That is, in an 
example in Fig. 6A, at the time of .electing the range of character string, 
the pointer 41 and the pointer 42 respectively designate a start point and 
an end point of the string. In an example in Fig. 6B, at the time of 
drawing a segment, the pointer 41 and the pointer 42 respectively 
25 designate a start point and an end point of the segment. In an example 

. 6C, at the time of cutting an object, the pointer 41 and the pointer 



42 designates » start point and an end point of a cutting line. 

The sixth manipulation example shown in Fig. 7 is an example 
which performs other manipulation simultaneous with scrolling. That .., 
while the dragging m.n.pulanon Is performed using the pointer 41. a 
scroll bar is manipulated using the pointer 42 so as to scroU a screen. 

The seventh manipulation example shown in Fig. 8 15 an example 
which designate, a position in a three dimensional virtual space. That is. 
an X coordinate and a Y coordinate are determined using the pointer 41 
cn an X-Y plane and a Z coordinate is determined by manipulating a shde 
bar using the pointer 42. The position of a pointer P inside the virtual 
space is determined based on these coordinate values. 

In any one of the above-mentioned manipulation examples, the 
user can efficiently perform the manipulation using both hands. This 
impU es that the manipulation environment which approximates the real 
S pace can be obtained even in the virtual space on the computer. 
Accordingly, the operability of the user is enhanced. According to the 
present invention, the manipulation can be performed with the feeling 
which approximates the manipulation In the real space. 



